Added CWLObjectType named union #220

GlassOfWhiskey · 2023-02-07T08:52:55Z

This commit introduces a CWLObjectType named union to represent a valid CWL object. Using a more precise union type instead of Any allows for a smarter parsing from SALAD auto-generated parsers. In particular, CWLObjectType is used to define the type of the default directive in CWL InputParameter and WorkflowStepInput fields. Plus, a CWLInputFile type is introduced to represent a CWL jobfiles as map<string, CWLObjectType>.

This PR depends on common-workflow-language/schema_salad#672.

mr-c

Alas, merging this would make significant changes in the HTML rendering of CWL v1.2[.1]

InputArraySchema now lists MapSchema and UnionSchema as acceptable items ; this is not okay!

Adding CWLObjectType

salad/schema_salad/metaschema/metaschema_base.yml

netlify · 2023-04-18T07:42:34Z

✅ Deploy Preview for cwl-v1-2-dev ready!

Name	Link
🔨 Latest commit	`9a5e721`
🔍 Latest deploy log	https://app.netlify.com/sites/cwl-v1-2-dev/deploys/655e6e3aef8288000816eb4a
😎 Deploy Preview	https://deploy-preview-220--cwl-v1-2-dev.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

mr-c

This looks good to me, thanks! We will need @tetron 's approval; and I still need to check the codegen & schema-salad changes

tetron · 2023-06-23T17:30:09Z

Introducing this to metaschema_base.yml means introducing new features to CWL, as Michael noted and as I noted over here common-workflow-language/schema_salad#672 (review)

Also, copying a note of that same ticket:

From the Introduction to 1.2:

"inherited records can narrow the types of fields if those fields are re-specified with a matching jsonldPredicate."

I forgot that we did that, but that might mean there is a way to exclude MapSchema and UnionSchema from polluting the CWL 1.2 spec.

If not, I think this needs to be a 1.3 change.

GlassOfWhiskey · 2023-11-22T15:36:51Z

@mr-c could you please take a quick look to the rendered doc and the directives just to check if you agree that there is something not working as expected? Because, if yes, maybe we can defer these directives to a future PR, after we fix the overall behaviour of this part of Schema SALAD

mr-c · 2023-11-22T15:43:09Z

I can not merge this into 1.2.1_proposed if it changes numbering for any existing documentation.

GlassOfWhiskey · 2023-11-22T16:03:17Z

But we said to merge it into the codegen branch if I remember correctly. Also because it requires Schema SALAD 1.3, which is not already out afaik

GlassOfWhiskey · 2023-11-22T16:08:29Z

In any case, probably the first PR that we should consider is common-workflow-language/schema_salad#672, as this one heavily depends on it

mr-c · 2023-11-22T16:12:04Z

But we said to merge it into the codegen branch if I remember correctly. Also because it requires Schema SALAD 1.3, which is not already out afaik

Yep, time to make a post-1.2.1_proposed codegen branch. You are right, that I will be less protective of the numbering there :-)

GlassOfWhiskey · 2023-11-22T21:22:52Z

@mr-c I changed base branch to codegen (just created it). The last two questions are:

How do we want to handle the dependency from the last schema_salad package? At the moment the requirements.txt file points directly to the PR, but I think it should point to a release (1.3-alpha1?)
Do you want me to keep the updates to the render.bash file? One day they will be migrated to the CWL website main branch, bu at the moment there is no Base.yml in the CWL main branch. Do you want me to replicate the Base.yml thing also in the main branch (the 1.2.1-proposed one) without the additional features to align the logic?

also run the tests against cwltool

This commit introduces a `CWLObjectType` named union to represent a valid CWL object. Using a more precise union type instead of `Any` allows for a smarter parsing from SALAD auto-generated parsers. In particular, `CWLObjectType` is used to define the type of the `default` directive in CWL `InputParameter` and `WorkflowStepInput` fields. Plus, a `CWLInputFile` type is introduced to represent a CWL jobfiles as `map<string, CWLObjectType>`.

CWL parsers should not follow `format` links.

mr-c · 2024-04-10T12:35:23Z

Do you want me to keep the updates to the render.bash file? One day they will be migrated to the CWL website main branch, bu at the moment there is no Base.yml in the CWL main branch. Do you want me to replicate the Base.yml thing also in the main branch (the 1.2.1-proposed one) without the additional features to align the logic?

Sure!

GlassOfWhiskey force-pushed the named-unions-refactoring branch 3 times, most recently from 0327a00 to 85a0cb5 Compare February 7, 2023 13:53

GlassOfWhiskey mentioned this pull request Feb 12, 2023

Added parsers for CWL input files common-workflow-language/cwl-utils#199

Merged

GlassOfWhiskey force-pushed the named-unions-refactoring branch from 9ec138c to fb45d9b Compare February 12, 2023 10:38

GlassOfWhiskey marked this pull request as ready for review February 12, 2023 10:48

GlassOfWhiskey force-pushed the named-unions-refactoring branch 2 times, most recently from 9a2db2e to 4347588 Compare February 12, 2023 11:02

mr-c requested changes Feb 15, 2023

View reviewed changes

salad/schema_salad/metaschema/metaschema_base.yml Show resolved Hide resolved

salad/schema_salad/metaschema/metaschema_base.yml Show resolved Hide resolved

GlassOfWhiskey force-pushed the named-unions-refactoring branch from 4347588 to 7ab983b Compare February 17, 2023 15:33

GlassOfWhiskey force-pushed the named-unions-refactoring branch 5 times, most recently from c429277 to c4d16f8 Compare April 18, 2023 10:13

GlassOfWhiskey force-pushed the named-unions-refactoring branch 11 times, most recently from d9cedc6 to 045c942 Compare June 18, 2023 18:49

mr-c approved these changes Jun 19, 2023

View reviewed changes

mr-c requested a review from tetron June 19, 2023 15:04

GlassOfWhiskey force-pushed the named-unions-refactoring branch 2 times, most recently from fae9a67 to 9a5e721 Compare November 22, 2023 21:10

GlassOfWhiskey changed the base branch from 1.2.1_proposed to codegen November 22, 2023 21:11

GlassOfWhiskey force-pushed the named-unions-refactoring branch 2 times, most recently from e91af6a to 8bb51c5 Compare November 22, 2023 21:16

mr-c and others added 13 commits April 10, 2024 15:32

Fewer containers (#129)

5351eee

also run the tests against cwltool

CI: less checks

5519ef9

Fix typo: $schema to $schemas

0857343

Fix a few typos (automatic IDE spellchecker)

941b65e

Moved CWLInputFile to Workflow.yml

b635305

Added flatten field to CWLObjectType array

4bbe625

Limited support fot MapSchema

aeb719c

Added Base.yml to render

ba0dadb

Removed DocType from Map and Union

b8c3152

Make CWLType extend original sld:Type

d6bc230

Added noLinkCheck to format field

e9acf61

CWL parsers should not follow `format` links.

set schema-salad minimum version

92c44c7

mr-c force-pushed the named-unions-refactoring branch from 8bb51c5 to 92c44c7 Compare April 10, 2024 12:35

ci: run on/for codegen branch

61699f5

mr-c approved these changes Apr 10, 2024

View reviewed changes

mr-c merged commit e7bd0ae into codegen Apr 10, 2024
3 checks passed

mr-c deleted the named-unions-refactoring branch April 10, 2024 12:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added CWLObjectType named union #220

Added CWLObjectType named union #220

GlassOfWhiskey commented Feb 7, 2023 •

edited

Loading

mr-c left a comment

netlify bot commented Apr 18, 2023 •

edited

Loading

mr-c left a comment

tetron commented Jun 23, 2023 •

edited

Loading

GlassOfWhiskey commented Nov 22, 2023 •

edited

Loading

mr-c commented Nov 22, 2023

GlassOfWhiskey commented Nov 22, 2023

GlassOfWhiskey commented Nov 22, 2023

mr-c commented Nov 22, 2023

GlassOfWhiskey commented Nov 22, 2023 •

edited

Loading

mr-c commented Apr 10, 2024

Added CWLObjectType named union #220

Added CWLObjectType named union #220

Conversation

GlassOfWhiskey commented Feb 7, 2023 • edited Loading

mr-c left a comment

Choose a reason for hiding this comment

netlify bot commented Apr 18, 2023 • edited Loading

✅ Deploy Preview for cwl-v1-2-dev ready!

mr-c left a comment

Choose a reason for hiding this comment

tetron commented Jun 23, 2023 • edited Loading

GlassOfWhiskey commented Nov 22, 2023 • edited Loading

mr-c commented Nov 22, 2023

GlassOfWhiskey commented Nov 22, 2023

GlassOfWhiskey commented Nov 22, 2023

mr-c commented Nov 22, 2023

GlassOfWhiskey commented Nov 22, 2023 • edited Loading

mr-c commented Apr 10, 2024

GlassOfWhiskey commented Feb 7, 2023 •

edited

Loading

netlify bot commented Apr 18, 2023 •

edited

Loading

tetron commented Jun 23, 2023 •

edited

Loading

GlassOfWhiskey commented Nov 22, 2023 •

edited

Loading

GlassOfWhiskey commented Nov 22, 2023 •

edited

Loading